pre processingbefor data mine algorithm target data set assembl data mine uncov pattern present data target dataset larg pattern remain concis mine accept timefram common sourc data datamart data warehous pre process essenti analys multivari dataset cluster data mine target set clean clean remov observ nois miss data clean data reduc featur vector vector observ featur vector summaris version raw data observ black white imag face px px bit raw data turn featur vector locat ey mouth imag reduc data vector bit code locat dramat reduc size dataset mine reduc process effort featur select depend object select featur fundament success data mine featur vector divid set train set test set train set train data mine algorithm test set verifi accuraci pattern found edit data miningdata mine commonli involv class task cluster task discov group structur data similar structur data classif task gener structur appli data email program attempt classifi email legitim spam common algorithm includ decis tree learn nearest neighbor naiv bayesian classif neural network support vector machin regress attempt find function model data error associ rule learn search relationship variabl supermarket gather data custom purchas habit associ rule learn supermarket determin product frequent bought inform market purpos refer market basket analysi structur data analysi edit result validationth final step knowledg discoveri data verifi pattern produc data mine algorithm occur wider data set pattern found data mine algorithm necessarili valid common data mine algorithm find pattern train set present gener data set call overfit overcom evalu test set data data mine algorithm train learnt pattern appli test set result output compar desir output data mine algorithm distinguish spam legitim email train train set sampl email train learnt pattern appli test set email train accuraci pattern measur email correctli classifi number statist method evalu algorithm roc curv learnt pattern meet desir standard reevalu chang preprocess data mine learnt pattern meet desir standard final step interpret learnt pattern turn knowledg 